Performance Evaluation Clinical Task Ontology(PECTO)

This poster presents a proposed Clinical Tasks Ontology (PECTO) designed to evaluate effects of technologies on human performance under controlled conditions, such as clinical simulation scenarios (CSS), across multiple clinical domains including prehospital care. In recent years there has been an explosion of technologies, including Information and Communications Technologies (ICTs) that are designed to assist health workers and improve their performance across a spectrum of clinical activities from pre-hospital care to post-surgical care. However, each new technology introduces its own requirements on the health worker and has the potential to either increase or decrease the perceived workload on the health-worker. Since perceived workload can have significant effects on health worker performance [4], it is important to carefully measure work-load changes and relate these to health worker performance measures such as task errors, and procedure compliance. Clinical Simulation Scenarios are often used to perform controlled experiments in which health workers’ performance on clinical conditions, simulated by various means, including Human Patient Simulators , is observed and measured with and without the technology being studied[5]. The Clinical Tasks Ontology (PECTO) was developed, among other applications, to help design such evaluation experiments. A major objective of such studies is to evaluate the performance of health workers as they perform specific clinical tasks. In this context, the PECTO presents a novel approach for task classification and analysis since previous approaches [6]–[8] do not account for sources of workload, and measurement of human performance in terms of errors and protocol compliance. An ontological approach was selected to build the classification system enabling tasks to have multiple properties that can be related to dependent variables. Previous work in task analysis and task classification include an ontological approach to plans and processes[6], some modeling of event evaluation [7], and a clinical task model of care plans[8], as well as comprehensive approaches to model human factors and workload. For the purpose of this research a Clinical Task (task) is defined as an action accomplished by a health care provider for the purpose of solving a clinical case. A case is a clinical situation that includes provider, patient’s conditions, clinical protocol or guideline to de followed and resources available. A case is expected to have a desirable outcome. Clinical Tasks can be determined by protocols. “Clinical protocols are agreed statements about a specific issue, with explicit steps based on clinical guidelines and/or organizational consensus. A protocol is not specific to a named patient”[6]. The PECTO developed here was part of a broader study focused on evaluating use on computerized clinical guidelines by community health workers[10], [11]. PECTO was constructed in Protégé by two clinical experts and is informed by several previous research studies on task analysis and clinical modeling. Each task in the PECTO has 9 possible properties classes, 75 distinct classes and 14 object properties. When applied to 30 pre-hospital cases for community health workers, following 6 clinical protocols, PECTO resulted in 447 identifiable individual tasks. In a study of task performance by Community Health Workers, application of PECTO enabled differentiation between learning and technology effects. Another application of PECTO is the development of a visual representation of case similarity. There are 5 object properties in addition to 9 basic object properties (relationships) between tasks and dimensions, as seen in Figure 2. PECTO object properties. These object properties allow for additional expressivity for particular inferred task, as a critical task (a task that is indispensable or its not execution ends in patient death). Or to establish complexity of tasks accordingly with number of subtask/goal Figure 1 PECTO Fully expanded Figure 2. PECTO object properties A. Extrinsic Evaluation For extrinsic evaluation a total of 982 tasks (individuals) were derived from 30 clinical cases. Each task was assigned at least a leaf class of each of 9 main domains. A total of 200 ontological distinctive task were obtained after applying reasoner. The most frequent task for the study particular data set was “Verify If Pulse is Present”, is it a task present in 20 out of the 30 cases. An ontological representation of such task is shown in Example 1. II. DISCUSSION AND CONCLUSION We developed a Clinical Task Ontology accounting with human performance factors. One limitation is that this first version is based on the clinical domain of pre-hospital care in which Community Health Workers are the primary clinicians. While this domain is very important in the global health context of developing countries, the developmental methodology can be extended to include other clinical domains . An important application, among others, of the PECTO is the ability to create metrics to compare cases from a human performance perspective. Future work include extending type of task model in order to have a reasoner-based classification of task depending on additional properties, instead of simply asserting the task type.